|
2021-09-14 16:02 developer |
cts_failure.jpg (185,599 bytes) |
|
|
Hi Jason I reported this issue to SKB. https://jira.skbroadband.com/browse/BFXBTF-280 I will let you know if there is any update. Thank you YK.Jung |
|
|
Hi YK, Wooshin, I re-verified this issue on my side tonight, and I think I should correct my description. Today, we compared the Pass report and the Fail report, and we found that the only difference is the NUGU_for_B_tv version. so I ask our QA to uninstall the NUGU_for_B_tv without rebooting and retest again. The fail item can be passed on both Eagle/Swan. But I tried this method on my side tonight, I still can NOT get pass result with this method. I try to analyze this issue. I pick up 1st failure and run the following command: run cts -m CtsAppSecurityHostTestCases -t android.appsecurity.cts.PermissionsHostTest#testCompatRevoked22 You can refer the source code of test item; https://android.googlesource.com/platform/cts/+/refs/tags/android-cts-10.0_r9/hostsidetests/appsecurity/src/android/appsecurity/cts/PermissionsHostTest.java (line: 164 and line 175) and refer https://android.googlesource.com/platform/cts/+/refs/tags/android-cts-10.0_r9/hostsidetests/appsecurity/test-apps/UsePermissionApp22/src/com/android/cts/usepermission/UsePermissionTest22.java (line 71) This test case is try to turn on/off Calendar permission in Settings UI. If the result fails, during the test, I found that focus will be lost, the calendar permissions can not be operated correctly. (Please check fail.png) Currently I found a way to pass this item, stay at login screen and open the Device information, then start the test. (Please see the infor.png) => The fail item can be passed. I have use this method to pass this failure more than 3 times, I think this is stable method, and I'll ask our QA use this method to verify this issue tomorrow. I also attach the PASS screenshot, as you can see the calendar permissions can be operated correctly. (Please check pass.png) Now I think this issue is not related to NUGU, it is more like a UI problem. (focus handling issue). But I have no idea now, why we can pass this item after I entering/staying Device information Page. I think we should check it with UI vendor. Thanks. pass.png (77,105 bytes) fail.png (80,061 bytes) info.png (113,762 bytes) |
|
|
Hi YK, Wooshin, Our QA used my method to retest these failures this morning. Test FW: 15.530.10 (eagle) / 1.531.16 (swan) Test steps: Step 0. Ensure that the system language is English Step 1. run the 1st failure, ensure that we can not get pass result in current state. Step 2. entered and stayed device information page Step 3. run the 1st failure, The fail item can be passed. reboot Step 4. run the 1st failure, ensure that we can not get PASS result in current state. Step 5. entered and stayed device information page Step 6. run the whole module, The 5 fail items can be passed. They also do an additional test. Test steps: Step 0. Ensure that the system language is English Step 1. run the 1st failure, ensure that we can not get PASS result in current state. Step 2. entered and stayed device information page *Step 3: Press BACK button to login page. Step 4. run the 1st failure, The fail item is still fail Currently, we can use "stay at device information page" to pass these items. You can check my method on your side. |
|
|
Hi, Jason, Could you please upload the log and result when the test was failed? Thank you. |
|
|
Hi Sangmin, Please check the attachments. |
|
|
Hi, Jason, If the STB has focus for entering certification number, these tests were failed.(screen01.png) And, when you press the KEYCODE_DPAD_DOWN and set the focus to the "비상설정", these test were passed.(screen02.png) I think the attribute was changed for waiting user input, so it affects the test results. I know the 3PL has only certificated STBs, so they can't see this screen. For internal test under Foxconn, you can pass this test after move focus. screen01.png (138,345 bytes) screen02.png (136,978 bytes) |
|
|
Hi Sangmin, OK, I'll ask our QA to verify your method, if we can pass these 5 failures stably with your method, I'll close this issue. By the way, 3PL always stays at login page for testing. Form Android P, TVStorm guided us to stay at login page for testing, because if we login to Home UI, the OTA pop up dialog or VOD may affect test. |
- yk.jung ( (ALTech) Younkwang Jung ) developer
